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Abstract — The bit-error threshold of the standard ensemble of 
Low Density Parity Check (LDPC) codes is known to be close to 
capacity, if there is a non-zero fraction of degree-two bit nodes. 
However, the degree-two bit nodes preclude the possibility of 
a block-error threshold. Interestingly, LDPC codes constructed 
using protographs allow the possibility of having both degree- 
two bit nodes and a block-error threshold. In this paper, we 
analyze density evolution for protograph LDPC codes over the 
binary erasure channel and show that their bit-error probability 
decreases double exponentially with the number of iterations 
when the erasure probability is below the bit-error threshold 
and long chain of degree-two variable nodes are avoided in 
the protograph. We present deterministic constructions of such 
protograph LDPC codes with girth logarithmic in blocklength, 
resulting in an exponential fall in bit-error probability below 
the threshold. We provide optimized protographs, whose block- 
error thresholds are better than that of the standard ensemble 
with minimum bit-node degree three. These protograph LDPC 
codes are theoretically of great interest, and have applications, 
for instance, in coding with strong secrecy over wiretap channels. 



I. Introduction 

Constructing a sequence of codes with efficient en- 
coders/decoders and a guarantee that block-error rate tends to 
zero with increasing block-length is one of the major goals of 
coding theory. At rates below capacity, such "good" sequences 
are known to exist, but many classical code sequences do 
not have this property. Modern code constructions, such as 
Low Density Parity Check (LDPC) codes, define a sequence 
of ensembles of codes with efficient decoders and probabilistic 
concentration results that come close to achieving the goal of 
constructing good code sequences |[T]- Recently, polarization 
Q and spatial coupling |3| have been used to construct good 
code sequences for binary symmetric channels. 

In this work, we are primarily interested in deterministic 
constructions of sequences of good LDPC codes with block- 
error thresholds nearing capacity limits. We will stick to the 
binary erasure channel, though the work can be extended to 
other binary-input symmetric channels. Most of the prior work 
in this area provides probabilistic guarantees on ensembles of 
LDPC codes, and most of these guarantees are for bit-error 
probabilities. The block-error threshold problem for LDPC 
codes was first studied in |4|, where a minimum bit-node 
degree (denoted ^min) of three was shown to be necessary for 
existence of block-error thresholds. For the standard irregular 



ensemble with Zmin = 2, the block-error rate, surprisingly, 
tends to a constant as block-length increases. The main cause 
for this problem is the presence of long chains of degree-two 
nodes in the standard ensemble. However, (bit-error) capacity- 
approaching LDPC degree distributions have a significant frac- 
tion of degree-two bit nodes. For instance, the best threshold 
for rate- 1/2 codes with minimum left degree three is only about 
0.461 leaving a significant gap to the capacity threshold of 0.5. 
So, while degree-two nodes are needed to approach capacity, 
they preclude the possibility of a block-error threshold. One 
of the goals of this work is to construct LDPC codes with 
block-error thresholds that improve this gap to capacity. 

A key idea in the construction of LDPC code ensembles 
with degree-two nodes and decaying block-error performance 
is the notion of multi-edge type (MET) ensembles |jTj, ||5), of 
which the protograph LDPC code ensemble |6| has received 
considerable practical attention because of ease of imple- 
mentation. In \5], the standard ensemble is restricted in a 
suitable fashion to limit the impact of degree-two nodes. In f6l, 
density evolution and optimization for protograph LDPC code 
ensembles was described and carried out. In protographs 
are optimized for thresholds nearing capacity, and ensemble- 
averaged weight distribution is used to establish block-error 
threshold for protograph LDPC code ensembles. There have 
been numerous other work in the construction of protographs 
in practical implementations. 

The use of large-girth graphs in constructing LDPC codes 
started with Gallager's thesis |8|, where regular LDPC codes 
with large girth were constructed. The Lubotzky-Phillips- 
Sarnak (LPS) constrction Q of Ramanujan graphs has been 
used in the construction of regular and irregular LDPC codes 
in pO) . As shown in |11|, large-girth LDPC codes with 

> 2, achieve an exponential decay 



minimum left degree, /ni 
of bit error, i.e C'(exp(- 



■cin'^^)) for constants ci, C2, over a 



binary erasure channel BEC(e), when e is less than the density 
evolution threshold e*. So, large-girth LDPC codes have a 
block-error threshold equal to their bit-error thresholds, when 

In this work, we provide deterministic constructions for a 
sequence of good LDPC codes by using large-girth graphs 
along with suitable protographs that contain degree-two nodes. 
This allows us to achieve BEC block-error thresholds as high 
as 0.486 with small (8 x 16) protographs. To do this, we begin 



by studying the density evolution for protograph ensembles, 
and show that bit-error decreases double exponentially in 
number of iterations at erasure probabilities smaller than the 
threshold even when Inun = 2, if long chains of degree-two 
variable nodes are avoided through the protograph. To avoid 
chains of degree-two nodes, we allow at most one degree-two 
variable node to connect to a check node in the protograph. 

We then provide a construction for large-girth protograph 
LDPC codes starting with a given large-girth regular graph and 
performing suitable node splitting operations. Using the LPS 
Ramanujan graphs, we provide deterministic constructions of 
large-girth protograph LDPC codes that achieve an exponen- 
tial decay for bit-error probability with blocklength and still 
contain degree-two bit nodes. In comparison with prior work, 
we have analyzed the density evolution for protograph LDPC 
codes directly and showed the double-exponential decay with 
iterations even in the presence of degree-two nodes. Our node 
splitting construction is more general than the one in | lOJ and 
provides a deterministic construction with guaranteed block- 
error probability behavior. 

II. Protograph LDPC Codes 

Following the notation in f^l, a protograph G = {V,C,E) 
is a bipartite graph with V and C being the sets of variable and 
check nodes, respectively, and E being the set of undirected 
edges that connect a vertex in to a vertex in C. Multiple 
edges are allowed between a pair of nodes {v,c) E V x C. 

A protograph can be represented by a base matrix B, where 
B{i,j) is the number of edges between the i-th check node 
(denoted Ci) and the j-th variable node (denoted vj). For 
example, consider a base matrix 



B = 



1 1 1 
1 1 1 
1 1 



The protograph corresponding to the above base matrix is 
shown in Fig. |l(a)| 

A. Lifted Graphs 

We can apply a copy and permute operation to a protograph 
to obtain expanded or lifted graphs of different sizes |6|. A 
given protograph G is copied, say T times, with the t-th 
copy having nodes {v,t) and {c,t), and edges {e,t). Then, 
for each edge e in the protograph, we assign a permutation 
TTe of the set {1,2, - •• ,T}. In the permute operation, an 
edge {e,t) connecting {v,t) and {c,t) is permuted so as to 
connect variable node {v, t) to check node (c, TTe{t)). We will 
denote the Hfted graph as G" = {V , C", E'). The Hfted graph 
of a protograph can be thought of as a Tanner graph of an 
LDPC code, which is referred to as a protograph LDPC code. 
In general, (T!)'^' lifted graphs or protograph LDPC codes 
can be obtained from a protograph, each corresponding to a 
different permutation, where \V\ is the number of variable 
nodes in the protograph. The collection of these lifted graphs 
is called the protograph ensemble of LDPC codes. Protograph 
LDPC codes are a special class of MET-LDPC codes, with 




(a) Example of a protograph. 




(b) Computation graph Ci{vi). 
Fig. 1. Protograph and computation graph. 



each edge in the protograph being of a different type. The 
degree distribution of check and variable nodes in the hfted 
graph is the same as that of the protograph. So, the (designed) 
rate of the protograph LDPC code is given by 1 — where 
|C| and \V\ denote the number of check and variable nodes 
in the protograph, respectively. 

Let ti be a variable node in the lifted graph G". The t- 
iteration computation graph associated with v, denoted Ct{v), 
is defined as the subgraph of G' obtained by traversing from v 
up to the t-th iteration level along all edges yj. The structure 
of the computation graph is completely determined by the 
protograph G for all lifted graphs G . An example of a 
computation graph is shown in Fig. 



1(b) 



B. Density Evolution for Protograph Codes 

Let us consider the standard message-passing decoder over 
a binary erasure channel (BEC(e)) run on a lifted graph G' 
derived from a protograph G = {V, G, E). Since the lifted 
graphs form an MET ensemble with \E\ edge types, density 
evolution proceeds with \E\ erasure probabilities, one for each 
edge in the protograph 1 12|. Let E = {ei, 62, 
edge e G E connecting variable node 
Ce- Let Xt{i) be the probability that an erasure is sent from 
variable node to check node along edge in the t-th iteration. 
Similarly, let yt{j) be the probability that an erasure is sent 
from check node to variable node along edge ej in the t-th 
iteration. The density evolution recursion |jT] is given by 

xo{i) = e, 



e\E\} with 
with check node 



yt+i 



(1) 



(2) 



where Ec{ej) = {i : = Cg.} is the set of all indices 
of edges adjacent to the same check node as the edge ej, 
and Ey{ei) = {j : Ve^ = Vej} is the set of all indices of 
edges adjacent to the same variable node as the edge e^. The 
density evolution threshold, denoted eth, for the protograph- 
based LDPC code ensemble is defined as the largest value of 
e for which erasure probability on each edge of the protograph 
tends to zero, as t — > oo. i.e. eth = sup{e : max,jXt(i) — > 
0}. Clearly, this is also the threshold value below which the 
erasure probability of a variable node in the protograph goes 
to zero with increasing iterations (as long as ^min > 1). 

C. Asymptotic Behavior of Density Evolution 

We now show that, for e < eth, xt{i) exhibits a double- 
exponential decay with t, when the number of degree-two 
variable nodes that are connected to a check node is restricted 
to one in the protograph. 

We will repeatedly use the following inequality. For any 
X e [0, 1] and a positive integer d. 



{d-l)x> l-(l-x) 



(3) 



Let d be the maximum degree of a check node in the 
protograph, and let xt = ma.XiXt{i). Since e < eth, we have 
Xt — ^ 0. We pick t large enough to have < {d ~ l)xt < 1. 
We note that 



Vi 



I - xtij) > I - Xt, Vj 

^ n ii-Mj))>ii~xtf''^ 

j6Bc(e,) 

Using ([TJ and ([3]), we get 

yt+i{j) < {d-l)xt, Vj 
^ xt+i{i) = e Y[ Vt+iU) 

eje_B„(e,) 

<e(d-l)('-i) {xtf-''l 
Repeating the process, we get 

yt+2{.r) < {d-i)xt+i{t) < e{d-iy- {xtf--'^ 

Xt+2 < e'™(d- 1) 



(4) 



(5) 



where l„i and /J„ are the minimum degrees of variable nodes 
in Ey{ei) and Ey{ei>), respectively. Since a check node is 
connected to at most one degree-two variable node, we have 
that either Im > i or l'^ > 3. So, (1^ - - 1) > 2. So, 

we have 



Xt+2 < A{xtf 



(6) 



where A = e'">(c? — i) does not vary with t. Since 

e < eth and xt 0, there exists an R such that Ax^ < 1 and 



{d— l)xfj < 1. Following arguments similar to those in |TT], 
we can show that 



XR+2i < (AxrY 



(7) 



which implies a double-exponential decay of xt with t (for 
sufficiently large t). 

For the standard ensemble of LDPC codes, density evolution 
analysis is approximate because of the following assumptions: 

1. The computation graph is a tree. 

2. The node degrees in the computation graph are indepen- 
dent. 

Typically, probabilistic concentration results and asymptotic 
guarantees are used to support the practical validity of density 
evolution in the standard ensemble. In this work, we use 
protograph codes and MET density evolution — these make 
Assumption 2 above unnecessary. For Assumption 1 to be true, 
we consider large-girth graphs. 

III. Large-Girth Protograph LDPC Codes 

To achieve strong secrecy. Tanner graphs whose girth in- 
creases as logn (though a growth of log log n is sufficient 
according to Section II-Ci with blocklength n have been 
used in fTT] . In this work, we extend the construction in 
|[TT| to construct bipartite graphs from protographs with girth 
increasing as log n. Once girth is 8(log n), message error rates 
for iterations up to 0(logn) will exactly follow the protograph 
density evolution of ([T])- (|2|. Following the analysis in Section 
II-C the message error rate falls double-exponentially in log n, 
or exponentially in block-length n. This results in an inverse 
polynomial decay, 0(l/n'') for any k, for block-error rate. 

A. LPS Graphs XP'I 

While the construction method can use any sequence of 
regular large-girth graphs, one explicit possibility is the LPS 
construction |9|. LPS graphs belong to the class of Cayley 
graphs. Given a group G and an inverse-closed subset S of 
G, i.e, s-i e S,ys G S, the Cayley graph (T{G, S)) is the 
undirected simple graph defined as follows: 

. The vertex set of T{G, S) is G. 

• For any g £ G and s E S, there is an edge between g 
and gs. 

Let p and q be distinct, odd primes with q > 2y/p. The LPS 
graph, denoted X'P'"^ |9|, is a connected, {p+ l)-regular graph 
and has the following properties: 

• If p is a quadratic residue mod q, then X^''^ is a 
non-bipartite graph with q{q^ — l)/2 vertices and girth 

>21ogpg. 

• If p is a quadratic non-residue mod q, then X^'"^ 
is a bipartite graph with q{q^ — 1) vertices and girth 

>41ogp9-logp4. 
When X^*'^ is non-bipartite, we can convert it to a bipartite 
graph using the following algorithm |11|: 

• Given a graph G with vertices V{G) and edges E{G), 
construct a copy G" with a new vertex set V{G') and a 



new edge set E{G'). Let / : V{G) V{G') be the 1-1 
mapping from a vertex in G to its copy in G'. 
• Create a bipartite graph H with vertex set V{G) U V{G') 
and edge set E{H) = {(xJiy)) : ix,y) e E{G)}. 

Following fT3), it was shown in |11| that g{H) > g{G). 

For constructing a sequence of d-regular large-girth graphs 
for an arbitrary d using the LPS graphs, we use the following 
trick from There exists an infinite number of primes p 
such that d\{p+l). For each such prime p and a suitable q, we 
construct XP''' and split each (p+l)-degree node into (p+1) /d 
nodes of degree d. As shown in pi] , the node splitting does 
not reduce girth and we have a large-girth graph of the required 
degree d. 

B. Node Splitting for MET-LDPC Codes 

The construction starts with a d-regular large-girth bipartite 
graph G with d being the number of edges in the protograph. 
The bipartition of G will contain |y(G)|/2 left and right 
vertices. We associate d sockets with each vertex of G, and 
associate each edge connected to a vertex with one of the 
sockets. 

According to Konig's theorem, the edge chromatic number 
of a bipartite graph is equal to the maximum degree of its 
nodes. Therefore G has an edge coloring involving d colors. 
Based on this edge coloring, we define a coloring of the 
sockets in G by the colors S = {si, S2, . . . , Sd}. Let P and 
Q be two fixed partitions of S, with P — {i^i, i^2, • ' ' ^z} 
and Q = {Qi, Q2, ■ ■ ■ , Qr}- If G is a Cayley graph, then the 
colors S can be associated with the generating set, along with 
a direction. 

The main step in the construction is splitting the left and 
right vertices of G according to P and Q, respectively. A 
left vertex v is split into sub-vertices vi,V2, ■ ■ ■ ,vi, such that 
for any i, the sockets of v in Pi get associated with Vi. A 
right vertex c is split into sub-vertices ci, C2, . . . , c,., such that 
for any j, the sockets of c in Qj get associated with cj. 
The resulting Tanner graph, denoted T(G, P, Q), will have 
Z|y(G)|/2 variable nodes and r|V^(G)|/2 check nodes, and 
the associated MET-LDPC code will have design rate 1 — j. 

The following observations summarize the important prop- 
erties of T{G,P,Q). 

1) It was shown in fTT) that the above node-splitting proce- 
dure does not decrease girth. So, the girth of T(G, P, Q) 
is not less than the girth of G for any P and Q. 

2) The Tanner graph T{G, P, Q) is, in fact, a lifted version 
of a protograph with I variable nodes indexed by Pi, 
1 < i < I, and r check nodes indexed by Qj, 1 < j < r. 
Variable node Pi in the protograph is connected by an 
edge to a check node Qj, whenever Pi n Qj ^ 0. So, 
the number of edges in the protograph is IS*]. 

3) The protograph is copied |G|/2 times and the edge 
permutation is induced by the edge connections of the 
original graph G. 

The procedure to generate a sequence of large-girth pro- 
tograph LDPC codes can be summarized as follows. By 
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Fig. 2. Illustration of (a) variable node splitting, and (b) protograph for |III-C| 



fixing the degree d and the partitions P and Q, we fix a 
protograph. We then apply the above node-splitting procedure 
to a sequence of large-girth d-regular bipartite graphs. This 
results in a sequence of large-girth Tanner graphs that are 
liftings of the protograph defined by (d, P, Q). 



C. An Example 

Let d — 12, and let the socket colors S = {si, Si, - ■ ■ 
be split into two partitions P and Q given by 

-Pl={si,S2} -P2 = {S3i ^41 S5} 

^3 = {S6,S7,S8} P4 = {Sg, SlO, Sll, S12} 



, S12} 



and 



Q\ = {si, S3, S6, sg, sio, sii} 

Q2 = {S2, S4, S5, S7, S8, ■S12} 



The variable nodes are split as shown in Fig. |2(a) The 
protograph generated by this choice of (d, P, Q) is shown in 



Fig. 2(b) The design rate of this protograph is 1/2. 

IV. Optimization of Protographs 
We have optimized protographs using differential evolution 



1 14 1 p3) , where we use the threshold given by density evolu- 
tion as the cost function. The salient steps of the differential 
evolution algorithm are described briefly in the following: 



1. Initialization: For generation G = 0, we randomly choose 
Np base matrices Bk.o, with < fc < Np — 1, of size 
|C| X \ V\, where Np = 10|C||y|. Each entry of Bk,G is 
binary, chosen independently and uniformly. 

2. Mutation: Protographs of a particular generation are in- 
terpolated as follows. 



k,G 



[S,,,G + 0.5(5, 



Br:i,G)], 



(8) 



where ri, r2, r3 are randomly-chosen distinct values in 
the range [0,Np — 1], and [x] denotes the absolute value 
of X rounded to the nearest integer 

3. Crossover: A candidate protograph B'^, q is chosen as 
follows. The entry of B'f. q is set as the 

th entry of Mk.c with probability pc, or as the (z,j)-th 
entry of Bf^ Q with probability 1 — Pc- We use p^ = 0.88 
in our optimization runs. In B'j. q, if any check node is 
connected to more than one degree-two variable node, 
edges are reassigned. So, each B'^. q avoids long chain of 
degree-two variable nodes. 

4. Selection: For generation G + 1, protographs are selected 
as follows. If the threshold of Bk^c is greater than that 
of set Bk,G+i = Bk,G; else, set Bk,G+i = Sfe,G- 

5. Termination: Steps 2-4 are run for several generations 
(we run up to G = 6000) and the protograph that gives 
the best threshold is chosen as the optimized protograph. 

Results from our optimization runs are given in Table [l] We 



Code type 


Threshold 


Standard ensemble (Imin = 3) 


0.461 


4x8 protograph in 




0.479 


8 X 16 protograph in 


10 


0.486 



TABLE I 

Optimized protographs and thresholds (rate 1/2). 



see that the optimized protographs give better thresholds than 
irregular standard ensemble codes with minimum degree 3. 
An optimized 4x8 protograph with threshold 0.479 is given 
by the following base matrix: 



(9) 



An optimized 8 x 16 protograph with threshold 0.486 is given 
by the following base matrix: 

"1 20010040000000 1' 
0100010022100011 
0312100040032203 
0500001100100100 
1311120010000000 
1500031000100000 
0400000110000001 
0500000001001010 

m 

Note that the above protographs have block-error threshold 
same as the bit-error threshold, and the block-error rate falls 



inverse polynomially in block-length under the large girth 
construction as described in Sections |ll]and|lll] In future work, 
we hope to obtain better thresholds that are closer to capacity 
limits, by further increasing the size of the protograph. 

V. Conclusion 

In this work, we presented a deterministic construction for a 
sequence of codes for the binary erasure channel with block- 
error rate falling inverse polynomially with block-length at 
rates close to the capacity. The codes are protograph LDPC 
codes that avoid long degree-two variable node chains and 
are constructed from large-girth graphs. To the best of our 
knowledge, this is the first deterministic construction of LDPC 
codes with guaranteed block-error thresholds nearing capacity 
limits. 
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